Exam AWS Certified Machine Learning - Specialty topic 1 question 1 discussion

The Answer is A. Reasons: 1. accurate is 86% 2. FN=4, FP= 10. The question is asking why this is a feasible model which means why this is working. So it is not asking the explaination of the unit cost of churn(FN) is greater than cost of incentive(FP). It is asking from the matrixs result, the number it self, FN(4) is less than FP(10). The model successfully keep a smaller number of FN regarding of FP.

Such question cannot be answered because we do not know how much more is greater the cost of churn than the cost of the incentive. CoC - Cost of Churn CoI - Cost of Incentive cost incurred by the company as a result of false positives = CoI * 10 cost incurred by the company as a result of false negatives = CoC * 4 So is it the case that CoI * 10 > CoC * 4 => CoI > 0.4 * CoC, or rather CoI < 0.4 * CoC? We don't know that because we don't know what does it mean "far greater", is it 100% greater, or is it 500% greater or any other number.

Should be A. Since the cost of churn is much higher, the priority should be focused on minimizing FN and a viable model should be one with FN < FP, isn't it?

Definitely C. If you look at the same question in https://aws.amazon.com/blogs/machine-learning/predicting-customer-churn-with-amazon-machine-learning/. Same question, but the confusion matrix is flipped in this case( TP top left, Tn bottom right) . When you miss an actual churn (FN) this would cost the company more. Therefore the answer is C 100%. I will die on this hill. I spent 20 minutes researching this to be certain. Most people who put A are incorrectly saying FPs are actual churns that are stated as no churn.. that is what a FN is. You can trust me on this.

There are more FP's than FN's, however the costs of FN's are far larger than that of FP's. So: numberof(FP) > numberof(FN), costperunit(FP) << costperunit(FN). This itself could suggest that totalcosts(FP) < totalcosts(FN), but would be somewhat subjective, since it is not stated how far the unitary costs are. What is suggested, however, is that the model is indeed viable (question asks WHY the model is viable, and not WHETHER it's viable). If the model didn't exist, there would be no way that there are FP's or FN's, but churns would still exist, which have the same cost as FN's. So it means the total costs with FP's must be less than the total costs with FN's (churns).

Correct Answer C. Explanation: The model's accuracy is calculated as (True Positives + True Negatives) / Total predictions, which is (10 + 76) / 100 = 0.86, or 86%. The cost of false positives (customers predicted to churn but don't) is less than the cost of false negatives (customers who churn but were not predicted to). Offering incentives to the false positives incurs less cost than losing customers due to false negatives. Therefore, this model is viable for production.

A. NO - accuracy is TP+TN / Total = (76+10)/100 = 86%; we know the model is working, so the cost of giving incentives to the wrong customers (FP) is less than the cost of customers we missed (FN), cost(FP) < cost(FN) B. NO - accuracy is 86%, precision is TP / (TP+FP) = 10 /(10+10) = 50% C. YES - accuracy is TP+TN / Total = (76+10)/100 = 86%; we know the model is working, so the cost of giving incentives to the wrong customers (FP) is less than the cost of customers we missed (FN), cost(FP) < cost(FN) D. NO - accuracy is 86%, precision is TP / (TP+FP) = 10 /(10+10) = 50%

C is the correct answer

FN has a higher cost than FP, so A is a better choice than C.

A) Because FN = 4 < FP = 10. FN are missed churns, and FP is misidentified churns.

A is the correct answer

cost of churn (churn cost) is greater than the cost of incentive (customers who do not churn)... the model predicts more false positives (customers who do not churn) than false negatives (customers who churn), Therefore, the costs of false negatives are greater than the costs of false positives, as churn is more expensive.

FN < FP

The question says "the cost of churn is far greater than the cost of the incentive", so we want to identify all the true churns, in order to do something about it. We don't want there to be any true churns we didn't see. This means we want false negatives as low as possible. So we want false negatives < false positives and we get exactly that in the model. Now this fact coupled with the fact that incentives are welcome rather than churn, in other words, cost / penalty for company is more when False Negative are predicted. So, Answer C - Cost incurred by the company as a result of False Positives is less than the False Negatives.

The closest answer to this rationale is: A. The model is 86% accurate and the cost incurred by the company as a result of false negatives is less than the false positives. Despite the answer options not matching the typical calculations of accuracy and precision, option A seems to be the most aligned with the company's goals if we consider the cost implications as more significant than the accuracy metrics alone. The company prefers a model has higher Recall score (10/14 this case 71.4%) than Precision score (10/20 this case 50%).

Cost incurred by the company is directly proportional to cost of churn which is directly proportional to number of false negatives. False positives are more acceptable than false negatives in this case.

accuracy is 86% so A or C. The cost of losing a customer is very high. Thus we do not want False Negatives (we do not want to predict no churn when there is churn). Thus the cost of a false positive is less than a false negative. Answer C